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We consider a network of coupled agents playing the Prisoner's Dilemma game, in which players 
are allowed to pick a strategy in the interval [0, 1], with corresponding to defection, 1 to coop- 
eration, and intermediate values representing mixed strategies in which each player may act as a 
cooperator or a defector over a large number of interactions with a certain probability. Our model 
is payoff-driven, i.e., we assume that the level of accumulated payoff at each node is a relevant 
parameter in the selection of strategies. Also, we consider that each player chooses his/her strategy 
in a context of limited information. We present a deterministic nonlinear model for the evolution 
of strategies. We show that the final strategies depend on the network structure and on the choice 
of the parameters of the game. We find that polarized strategies (pure cooperator/defector states) 
typically emerge when (i) the network connections are sparse, (ii) the network degree distribution 
is heterogeneous, (iii) the network is assortative, and surprisingly, (iv) the benefit of cooperation is 
high. 

We study a Prisoner's Dilemma game on a network to uncover the effect of the network structure 
on game dynamics. We describe a model for an individual's changing strategy based on a payoff 
comparison with the player's neighbors. This type of model is relevant to many situations where 
strategies change due to payoffs such as in politics, economics, or finance. 

I. INTRODUCTION 

The Prisoner's Dilemma is a paradigmatic model for interactions among agents where strategies leading to either 
individual gain (defection) or the common good (cooperation) are in competition. In politics, economics, and in 
finance, an individual's actions may be regarded as cooperation or defection. Sometimes, the choice of a strategy 
(whether to cooperate or defect) is affected by direct reciprocity but other times, the choice of strategy is affected by 
the performance of players in the game. 



Previous research has focused on how direct reciprocity in a repeated game (i.e., trust between connected individuals) 
can affect the evolution of strategies flfl . Alternatively, players' performance may affect their goals and future choice 
of strategies. For example, direct reciprocity cannot capture altruistic or selfish behavior. Therefore, a problem of 
interest is understanding, when the players are coupled in some complex way over a network, how their strategies 
evolve in time, based on their performance in the game. This is the subject of the current paper. 

Evolutionary game theory studies how players' strategies evolve when the game is iterated in time. Here we consider 
that the agents are coupled over a complex network and that the game is played between each agent and his/her direct 
neighbors over the network. An important difference with previous literature is that our analysis is based neither on a 
population level analysis 



42[ nor on a mean field approximation; instead, we are interested in how a specific network 
structure affects the evolution of strategies, and in particular the fixed points of the dynamics and their stability. For 
a review on evolutionary games on networks, see 6|. 

In recent years, much research effort has been devoted to studying the structure and dynamics of complex networks 



(see e.g., 7H18j). However, a full understanding of how the specific network structure affects the evolution of strategies 
for models in evolutionary game theory is lacking. Here we present a unified analytical treatment which holds for 
diverse networks including those with random, scale free, and degree-correlated topologies. 

In the literature on game theory, a distinction is commonly made between pure and mixed strategies. In the first 
case, players are allowed to choose from a finite set of alternatives (e.g., in the case of two strategies, either defection 
or cooperation). In the second case, each player is allowed to choose a strategy in a continuum range, e.g., [0,1], 
with (1) representing pure defection (pure cooperation). In a probabilistic framework, a mixed strategy equal to 
0.25 corresponds to cooperating 1/4 of the time and defecting 3/4 of the time. In many real social situations, mixed 
strategies are common, as rarely an individual is observed to behave as a cooperator (defector) all of the time. It is 
also reasonable to expect that mixed strategies will sometimes be convenient, as they allow a player to hedge against 
the risk of choosing a pure strategy (in the same way as diversifying a portfolio reduces the risk of an investment). 
In what follows, we will consider a network of coupled agents, each of which can choose a strategy in a continuum of 
values between pure defection and pure cooperation. Agents are then able to update their individual strategy based 
on the interactions with the neighbors. Pure strategies are eventually recovered as particular cases of mixed strategies 
(i.e, or 1). Hence, the pure strategies lie on the boundary of the mixed strategies. As a result of this, we will see 
that pure strategies play a particular role (qualitatively different) in the dynamics. 

In what follows, we introduce a model for the evolution of strategies that we analyze. The main features of our 



model are the following: I) it is payoff driven; II) each agent has limited information on the other players' strategies. 
Below we describe points I) and II) in more detail. 

I) Most of the ongoing research on evolutionary game theory focuses on situations where the agents are equal, i.e., 
they respond in the same way when presented with the same stimulus. However, this assumption is unrealistic. In this 
paper, we model a situation where each node's choice of strategy depends on an internal parameter that may differ 
from node to node. We want this parameter to evolve with time based on the interactions with the other players. We 
consider that each individual's choice of strategy depends on his/her degree of success in playing the game; specifically, 
we consider that the level of accumulated payoff at each node becomes a relevant parameter. As motivation, we note 
how, in social contexts, the behaviors of individuals are highly correlated with their social status; also, in economics 
powerful agents/players are usually more competitive than weaker ones. Maslow's theory of motivation 19] ranks 
human needs in a hierarchy (usually represented as a pyramid) and argues that as each lower need is satisfied, the 
next higher level becomes more compelling, i.e., there is a shift in the perception of the relevant needs at each level 
of the hierarchy. In what follows, we will incorporate this feature in our model of an iterated game, i.e., once the 
accumulated payoff of player i exceeds a threshold, this will have an effect on the choice of strategy of i. 

II) Most of the current research in evolutionary game theory considers that individuals choose their strategy based 
on knowledge of other players' strategies. However, this assumption is unrealistic. In this paper, we consider a 
situation in which knowledge about other players' strategies may be unavailable, while we assume that each player 
has knowledge of his/her neighbors' payoffs. These assumptions apply to those situations in which the strategies that 
players adopt are hidden or undeclared. For example, we may expect that in certain situations a defector may be 
reluctant to openly declare his/her strategy. Note that if information on the payoffs is available, this can provide 
indirect evidence of the strategies adopted by the players; i.e., for example, if a neighbor is driving a Ferrari, this 
might indicate that he or she must be a strong cooperator or a strong defector; but which one would be the right 
guess? 4^]] 

To conclude, our model presents both these characteristics: it is payoff-driven and it does not presume knowledge 
of the other players' strategies. As such, it aims at encompassing certain sources of complexity that characterize 
real-world situations. Given our model, we are able to carry out a stability analysis that is highly dependent on the 
specific network structure and provides a unified framework to study the effects of different network features, such as 
random, scale free, and degree-correlated topologies. 

In Sec. II we present our model for the evolution of strategies of coupled agents over a network. In Sec. Ill we 
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classify the fixed points of the dynamics in fully mixed strategies and polarized strategies and we present a stability 
analysis that gives a simple condition on the eigenvalues of a relevant matrix. The role of the underlying network 
topology on stability is considered in Sec. IV. Conclusions are presented in Sec. V. 



II. MODEL FORMULATION 

We consider a network of coupled agents (nodes) playing the Prisoner's Dilemma game. Each node i = 1,...,N 
is characterized by a strategy <7j in the closed interval [0,1], where the strategy <7j represents the probability that 
player i behaves as a cooperator over a large number of interactions (1 — a is the probability that player i behaves 
as a defector). The case of cti = (ai = 1) corresponds to a pure strategy, in which player i is always a defector 
(respectively, a cooperator). We refer to all the remaining cases, i.e., Ci 6 (0, 1), as mixed strategies. We assume that 
the game is played very frequently so that at our resolution we are unable to assess whether agent i behaves as a 
cooperator or a defector at any given interaction, but only the average number of times that it acts as a cooperator, 
i.e., the frequency <7j, which we regard as the state of node i. Hence we assume to only have access to the probability 
Cj (while information about whether a node behaves as a cooperator or defector at any given interaction remains 
unavailable), and we regard cr, as the state of node i. Over many interactions, individuals may occupy any position 
in the range of probabilistic defector/cooperator behavior when considering their record of behavior. 

We consider that the evolution of the strategies is governed by the nodes' payoffs, where the payoff of node i is 
determined by its strategy and those strategies of the nodes that are connected to it. In the Prisoner's Dilemma (see 



e.g., Ref. 



20 



21]), a cooperator incurs a cost c for each connection that he or she has. If a cooperator is connected 
to another cooperator, the benefit of that connection is b. So, if a cooperator is connected to n individuals and m 
of them are cooperators, the payoff for that cooperator is bra — cn. Typically, b > c. For a defector, there is no cost 
associated with any connection, and if the defector is connected to I cooperators, the defector's payoff is bl. 

We assume that the network structure could be directed and weighted and is described by the adjacency matrix 
A = {Aij}, where Aij > if j is connected to i, otherwise. This yields the following definition of the payoff for a 
player i, 

Pi = a-i[b^2 A io a 3 ~ c^Aij] + (1 - a^b^Aijdj = b^A^aj - c^A^ = ^A ij {b<r j - an), (1) 

3 3 3 3 3 3 

or in matrix notation, 

p=(bA- cD)a, (2) 
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where p — \p\,p2, ■■■^Pn] is the vector of payoffs, a — [<Ti, 02, crjv] is the vector of strategies, and 13 = {<%} is 
a diagonal matrix, such that djj = ■ A^ . Again, pi in ([T]) is the average payoff of node i over a large number of 
interactions with its neighbors, when its strategy and those of the neighbors are given by fixed cr's. 

We allow the strategies adopted by each node/player to evolve in time; thus we write Ui = <7i(t), where t represents 
continuous time. Our basic assumption is that each node updates its strategy based on its payoff and the payoffs of 
its neighbors (note we do not make the assumption that nodes know the strategies of their neighbors. Indeed, that 
knowledge could be unavailable). With this information, player i can compute the relative payoff, 

u t = J2MMt)-Pj(t)l (3) 

3 

with a positive (negative) value of Ui indicating that node i is performing better (poorly) with respect to its neighbors. 
From (|2I3|) . we obtain, 

u = (D — A)p = Ba, (4) 
where u — [u±, 112, un], and the matrix B is defined as, 

B = (D -A)(bA-cD). (5) 

From we now see that the relative payoff of node i can be written as a linear combination of the strategies, 
i.e., Ui = BijCTj, i = 1,... } N. Note that the matrix B encodes information of both the network structure (i.e., 
the matrix ^4) and the choice of the parameters of the game (i.e., the two scalars b and c). We will show that the 
eigenvalues of this matrix control stability of a fixed set of strategies. 

We aim at formulating a simple, general model for the evolution of the strategies {<7,(i)}, evolving from tTi(O) € (0, 1), 
that fulfils the following requirements, 

(i) The pure strategies Oi — {0, 1} are fixed points for the dynamics, as <Ji(t) cannot decrease below or exceed 1. 

(ii) The dynamics admits other fixed points for values of <ii G (0,1), corresponding to mixed strategies in the 
spectrum of possibilities between complete cooperation and complete defection. 

(hi) The stability of the fixed points [(i),(ii)] depends on an internal adaptive parameter at each node (which we 
label fii, i — 1,..,N) whose evolution is governed by the interactions with the neighboring nodes. Under specific 
conditions, [ii is a measure of the wealth of i, i.e., the accumulated payoff at node i. We assume that when /ii exceeds 
a threshold, this will affect the choice of strategy of i. 
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FIG. 1: The table (on the left) shows the stability (S) or instability (U) of each of the fixed points {0, l,fa} for any occurrence 
of fa. Note that the stability of the fixed points <7; = fa for fa > 1 and for fa < is in brackets as these equilibria are never 
observed if Ci(0) 6 (0, 1). On the right, we have a graphical representation of the stability of the fixed points of Eq. (I6a|l for 
different values of fa, where open circles represent unstable fixed points and filled circles represent stable ones. From top to 
bottom, we show the cases of fa < 0, < fa < 1, and fi > 1. 

With these conditions in mind, we write the following set of differential equations, 

&i(t) =aa i (t){a i {t) - l)(o*(t) - fa(t)), (6a) 
£i(t) =/(«i(*)), (6b) 

i = 1, ...,N. Equation (|6a|) determines the evolution of the strategies, where a > 0, fa is the adaptive parameter, 
whose evolution is specified by Eq. (|6b|) . Here, /Xf is the z th node's internal parameter that models the player's 
changing attitude toward his neighbors. The evolution of fa[t) is forced by the input fait) = J^ - Bij<jj(t), through 
the coupling function / that we require be continuous and strictly monotonically increasing/ decreasing with /(0) = 0. 
For f(u) = u, fa is simply the integral of Ui(t), the relative payoff of node i, over time, and as such it represents the 
relative wealth of node i (see (hi) above). Here, again, the word relative, means with respect to node i's neighbors. 
We wish to emphasize that from Eq. ([5]), our model is payoff-driven (it evolves based on a comparison of the payoff 
between node i and its neighbors) and does not presume knowledge at node i of its neighbors' strategies Oj, j =/= i. 

The parameter fa determines the stability of the fixed points of Eq. (|6aj). The differential equation (|6a|) has three 
fixed points: = {0, l,fa}- The form of (pa]) is such that for any initial condition (0) £ (0, 1), the trajectories are 
constrained to lie in the closed interval [0, 1]. We now evaluate stability of the fixed points of (|?5al) . when the values 
of fa are fixed. This is summarized in Fig. 1. For fa £ (0, 1), the pure strategies a% — {0, 1} are unstable, while the 
mixed strategy <n — fa is stable. For fa > 1 (fa < 0), the fixed point 1 is stable and the fixed point is unstable (the 
fixed point 1 is unstable and the fixed point is stable). 



When ^ increases (decreases) above 1 (below 0), it affects the stability of the fixed points of Eq. (j6a]) as shown in 
Fig. 1. Specifically, for f(u) = u, when fa exceeds (goes below) a threshold, this results in a qualitative change of the 
behavior of i. This corresponds to a transition from one level to another of the Maslow's pyramid of needs, resulting 
in a change of the individual's attitude (and, as a consequence, his/her strategy). 

In our numerical simulations to follow, we choose the following form for the function /, 

/(u)=0tanh(7u), (7) 

/3, 7 > 0. With this choice, fa can still be seen as the wealth accumulated at node i but with a saturation on the 
maximum increment or decrement that is allowed. There are many possible situations in which gains and losses are 
typically bounded. As an example, consider a taxation system that charges a tax on capital gains, while capital losses 
are tax-deductible. 

The form of the function / in ||7J is consistent with the observation that favorable (unfavorable) conditions often 



foster cooperative (defective) behavior 
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-24] (e.g., empirical studies have found a positive correlation between the 

n 

level of cooperation and the payoff for players participating in a Prisoner's Dilemma experiment 22]). However, we 
wish to emphasize that the results that we present in this paper (and in the particular, the stability analysis of Sec. 
Ill) are, to a great extent, independent of the specific choice of the function /. On another note, we observe that the 
function / describes how the relative payoff influences the strategy adoption, hence it defines an imitation function 
as defined in Rcf. 



ILL STABILITY ANALYSIS 

In this section we consider the stability of the fixed points of Eqs. ©, when the parameters fa are not fixed (i.e., 
the individual systems i = 1,...,N are coupled). We first look for fixed points {o~*}^ =1 of the dynamics for Eqs. 
(|6|). A candidate fixed point {c r i}fL 1 may include both pure and mixed strategies for different nodes, i.e., for some 
i's Oi = {0, 1}, and for the remaining i's, ai = fa G (0, 1). We refer to a fixed point for which a* € (0, 1), for all 
i = 1, ...,7V as a fully mixed strategy, otherwise, we refer to it as a polarized strategy. 

In what follows, without loss of generality, we assume that for any equilibrium point, the indices i are labeled so 
that a* = {0, 1}, i = 1, ...,£, < £ < N, and a* € (0, 1), i = £ + 1, ...,N. With this relabeling, we note that the 
equations (foa| . i = 1, ...,£, are decoupled from the corresponding equations f|6b[) . i = 1, ...,£. It follows that the first £ 
equations ^ drive the last (N — £) equations ©. Thus we define any set of strategies to be a fixed point {<J*}^Li if 
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the following conditions are met: (I) Eqs. (l6al) arc set to zero for i = !,...,£ and (II) Eqs. (|6al6bl) are simultaneously 
set to zero for i = £ + l,...,N = a* , i =£ + l,...,N). 

Given a vector of strategies, {0i}£L 1; we would like to derive conditions on this vector that identify it as a fixed 
point. Conditions (I) are always satisfied if <Ti G {0, 1} for all % = 1, ...,£. Conditions (II) become, Ylj=t+i Bij&j = 
— ^2j—l BijOj, i = £ + 1, N, or in matrix form, 

B r a r = c, (8) 

which is a system of (N — £) linear equations in (N — £) unknowns; the matrix B r is a reduced matrix, which is 
obtained by eliminating the first i rows and the first i columns from the matrix B, a r is an (N — £)-vector which 
is obtained by eliminating the first £ entries from the vector a, and c = [ci, C2, Cff—i] is an (N — £)-vector, which 
is determined by the first £ entries of a, i.e., q = — 53j=i ■^ij CT j- If 5 = 0, the system of linear equations © is 
homogeneous (case (A)), so any a r in the null subspace Ker(£? r ) of the matrix B r is a solution. Otherwise, ((8} is 
an inhomogeneous system (case (B)) and it admits one and only one solution, provided that B r is invertible, i.e., 
o r — (B r )~ 1 c. Fully mixed strategies (for which £ = 0, B r = B) are included in case (A)|44|. 

We consider <Ji(0) £ (0, 1), which constrains ai(t) to be in the interval [0, 1]. A vector in Ker(£> r ) does not necessarily 
have to represent a set of physical strategies. For example, for a given matrix B r , it is possible that there are vectors 
in Ker(B r ) that have both positive and negative entries. Then, even with the multiplicative constant that we are 
allowed for eigenvectors, it might be impossible to rescale all entries of such vectors into the range [0, 1]. Thus, in 
order for a r to be a physically meaningful equilibrium for the set of equations (|6al) . i = £ + 1, N, it has to satisfy 
the additional requirement (III) that: for case (A), all the entries of a r have the same sign (i.e., they are all positive 
or all negative) and, for case (B), the entries of the vector a r are in the interval (0,1). For the remainder of the 
paper, we assume the dimension of Ker(B r ) to be 1; in which case, the single zero eigenvector vq of B r is the basis 
for Ker(£? r ). Then any vector a r = clvq, where a £ I is a fixed point but not necessarily physical. 

We see that the system of equations ^ behaves as a multistable system for which the final attractor depends on the 
choice of the initial conditions. Here, we include a stability analysis for the fixed points of (O, i.e., for points {cr*}fL 1 
that satisfy the above conditions (I), (II), and (III). We already know that the first £ equations (pa| are decoupled 
from the others and admit only one stable equilibrium (if is stable, 1 is unstable, and viceversa). Therefore, in order 
to study stability of an equilibrium {<J*}^ =1 , we need to linearize Eqs. (|6b|) . for i = £ + 1, N, about {(J*}fLi (with 



/4 g (o, i), « = ^ + 1, jv), 



S&i =a(a* 2 - <r*)(8a l -8^), 



N 



8fn=Df(0) B ufoh 



(9a) 
(9b) 



3=1+1 

i=£ + l,...,N, where, since < a* < 1, we have that -0.25 < (a* 2 -a*) < 0. We now write Ylf=e+i B i3^ a i = XkSai, 
i = £ + 1,...,N, where Xk is an eigenvalue and [8ai + i, <5ctjv] is an associated eigenvector for the matrix B r , 
k = 1, (N — £). By substituting in (|9b|) we obtain, 



8& l =a(a* 2 - <r*){8a l -8^), 
8{ii =Df(0)\ k 8ai, 



(10a) 
(10b) 
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i = £ + 1, AT, and fc = 1, (AT — £). Note that Eqs. (jTOJ) represent a set of (N — £) 2 equations, each of which is 
independent of the others. Equations (|10p can be rewritten, 

—nd>A 8ui 

(11) 

i=£ + l,...,N, k = l,...,(N-l), where & = (cr* 2 -of) < and A fe = A£ +j'A|, where j 2 = -1 

We first proceed under the assumption that the A^'s are real (which is the case e.g., when the matrix A is symmetric, 
i.e., the network is undirected, see Appendix B). We see from (ITTj) . that stability is independent of 4>i, a s long as <f>i 
is negative, but depends on the conditions, 



£>/(0)A fe <0, k = l,...,(N-£). 



(12) 



The case of Xk = corresponds to 8a decaying to zero, with 8fi constant. Hence, this A^ ~ eigenvalue corresponds 
to neutral stability along the direction of the eigenvector vq. For Df(0) > (Df(0) < 0), condition (fT2|) is equivalent 
to X max < (X min > 0), where X max = max fc=lj ...,(jv-«) ^fe (Kiin = min fc=li (A r_ £) ). Note that this is a necessary 
and sufficient condition for stability. 

In the case the eigenvalues Afc of the matrix B r are complex, stability requires that the real parts of both the 
eigenvalues of the system (fTTj) be negative for i = £ + 1, N and k = 1, (A" — I). This corresponds to the following 
conditions, 



Df(0)X r k < 0, 
|D/(0)A l fc | < y/afcDmxt, 



(13a) 
(13b) 
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FIG. 2: We consider an 8-node network, shown in plot d. The benefit of cooperation is set to b = 1.1 (c = 1). Plot a shows the 
time evolution of <Ji(t), i = 1, . . . , 8, with all the strategies cn(t) converging on a fully mixed state, corresponding to vo. Plot 
b shows the evolution of fn(t), i = 1, . . . , 8. Plot c shows the evolution of Pi(t), i = 1, . . . , 8, with all the payoffs converging 
after a transient to 0.165. The initial conditions for /i's and <r's are randomly chosen from a uniform distribution in the interval 
(0, 1), a = 5 x 1(T 2 , /3 = 10" 3 , 7=1. 

which need to be satisfied for i = I + 1, ...,N and for k — 1, (N — £). In what follows, we proceed under the 
assumption that the eigenvalues are real. 

One nice property of the stability condition (|12p is that it decouples the effect of the coupling function / from the 
effects of the eigenvalues Xk of the matrix B r (of the matrix B in the case I = 0), where the eigenvalues Xk reflect the 
particular choice of the network structure (i.e., the matrix A) and the form of the game (i.e., the parameters b and 
c). Also, condition (1121) allows us to predict whether some network/game combinations would support fully mixed 
strategies or promote polarized behaviors. For example, given the network in Fig. 2(d), we are able to determine that 
for b = 1.1 there is a stable fully mixed strategy and for b = 1.5 it is unstable. 

Figures [2] and [3] show the effect of the choice of the parameters of the game on the stability of the strategies. Fig. [2] ([3]) 
shows a simulations for b = 1.1 (b = 1.5), where b is the benefit of being connected to a cooperator (c = 1 in both cases). 
For b = 1.1, the eigenvalues of the matrix B are {-38.7, -20.2, -12.4, -9.15, -2.42, -1.00, -0.0841, 0}, i.e., they are all 
negative. The eigenvector associated with the eigenvalue is v = [0.405, 0.260, 0.194, 0.194, 0.409, 0.423, 0.423, 0.409], 
i.e., its components are all of the same sign. Thus we expect a fully mixed strategy proportional to vq, to be a stable 
fixed point for the dynamics. The initial conditions (7^(0) and are randomly selected from a uniform distribution 
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FIG. 3: We repeat the simulation in Fig. 2, but now the benefit of cooperation b — 1.5. With this choice of the parameters, 
the fully mixed state is unstable. Plot a shows the time evolution of Ui(t), i = 1, . . . , 8. Nodes 3 and 4 (shown as squares in the 
graph) converge on the defector state (0); nodes 6 and 7 (shown as circles in the graph) converge on the cooperator state (1); 
the remaining four nodes (shown as triangles) converge on a mixed strategy. Plot b shows the evolution of /x;(i), i = 1, ... ,8. 
Plot c shows the evolution of the payoffs Pi(t), i — 1, ... ,8, which converge on different final values: pi = 1.13, P2 = 0.652, p3 = 
0.411, p4 = 0.411, = 1.21, pe = 1.29, P7 = 1.29, p$ — 1.21. The initial conditions for /Vs and cr's are randomly chosen from a 
uniform distribution in the interval (0, 1), a = 5 X 10~ 2 , ft = 10~ 3 , 7=1. 

in the interval (0, 1). Fig. [2fa) ([2jb)) shows the evolution of the individual ai{t) (of the /ij(t)) versus t, with the Ui 
(and the //j) converging on a solution proportional to vq. Fig. [2jc) shows the time evolutions of the payoffs Pi(t), 
i = 1, N , which after a transient converge on the same value Pi = 0.165, i = 1, N . 

For b = 1.5, the eigenvalues of the matrix B are {-42.9,-22.9,-13.5,-9.74,-2.06,-1.00,0,0.148}, and v Q = 
[0.203,0.562,0.556,0.556,0.0706,0.0824,0.0824,0.0706]. Since one of the eigenvalues is positive, we do not expect to 
see a solution proportional to Vq. The case of b — 1.5 is shown in Fig. [3l As can be seen from Fig. [3ta), after a 
transient, nodes 3 and 4 (shown as squares in the graph in Fig. [3Jd) ) converge on the defector state 0; nodes 6 and 
7 converge on the cooperator state 1 (shown as circles). The remaining four nodes converge on a mixed strategy 
(shown as triangles). The eigenvalues associated with the reduced matrix B r obtained by eliminating the third, 
fourth, sixth, and seventh rows/columns from the matrix B, are {—40.1, —10.0, —5.50, —4.40}, which ensures stability 
of the solution a r = B r ~ x c = [0.983,0.274,0.880,0.880]. Fig. rjjb) QJfc)) shows the evolution of the individual 
/Ltj(t) (of the payoffs Pi(t)). In contrast to the case of b = 1.1, the nodes' payoffs converge on different final values: 
pi = 1.13, p 2 = 0.652, p 3 = 0.411, p 4 = 0.411, p 5 = 1.21, p 6 = 1.29,p 7 = l-29,p 8 = 1.21. The different final payoffs of 
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in the network. Note that we have repeated 



the players for b = 1.5 are to be ascribed to their different locations 
both the simulations in Figs. [2] and [3] many times, starting from different sets of initial conditions, and always observed 
convergence on a solution proportional to Vq . 

It is evident that fully mixed strategies (the fixed point observed in Fig. [5]) and polarized strategies (the fixed point 
observed in Fig. [3]) are substantially different. In particular, there are two main qualitative differences between the 
two, 

1. Fully mixed strategies are fixed points for which the payoffs are equal at all the network nodes, while polarized 
strategies are fixed points for which the payoffs are uneven. 

2. Fully mixed strategies are sensitive to the form of the network and the parameters of the game. Therefore it is 
possible to adjust/control a fully mixed strategy by changing the network connections and/or the parameters 
of the game. However, for polarized strategies, there are some players whose strategies cannot be changed, and 
thus the full state cannot be adjusted. 

Fully mixed strategy represent a fair situation, in which all the nodes achieve equal payoffs, regardless of their 
location in the network. On the other hand, polarized strategies represent an unfair situation, in which the nodes' 
payoffs vary according to their respective locations in the network. Therefore, it becomes important to understand 
how parameters of the game and variability of the networks yield stability of the fully mixed strategy. 

The effects of varying the parameter b (benefit of mutual cooperation) on stability of the fully mixed strategy are 
reported in Fig. 4. Figure 4(a) shows the result of evolving the network in Fig. 2(d) for a long time from random 
initial conditions in the range (0,1) as a function of the game parameter b for c = 1. At the end of each run we record 
the vector of the final strategies aj and we plot the angle A between the final state 07 and the state associated with 
the eigenvector %, 

A = tt" 1 arccos ( f _° ) , (14) 

where we have indicated with the symbol, ■, the dot product between vectors and with || ■ || the Euclidean norm of 
vectors. A is close to zero for b < 1.15 and grows for larger b. This is predicted by our stability criteria (see Fig. 4(b)). 
In Appendix C we show that in the limit in which b — c, the fully mixed strategy is stable. We expect this property 
to hold for b close to c. However, for b above a critical value, the fully mixed state becomes unstable. This is shown 
in Fig. 4(d) where the largest nonzero eigenvalue of the matrix B is plotted versus b. The dashed section of the curve 
(starting at b ~ 1.15) represents the range over which the entries of the eigenvector vq are not all of the same sign. 
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FIG. 4: Plot (a) shows the result of evolving the network in Fig. 2d for a long time from random initial conditions in the range 
(0, 1). We plot A, the angle between the final state a; and the state associated with the eigenvector vo as a function of the 
game parameter b for c = 1. The label R represents the range of stability. Plot (b) shows the largest nonzero eigenvalue of the 
matrix B versus b. The dashed section of the curve represents the range over which the entries of the eigenvector wo are not 
all of the same sign. 

Thus wc define the range of stability of a fully mixed strategy, R, as the range in the game parameter b (for b > c) 
such that the largest nonzero eigenvalue of B is negative and the entries of the eigenvector w are of the same sign. 

An interesting result of Figs. 2, 3 and 4 is that by increasing the benefit to cost ratio b/c, the fully mixed state 
may be destabilized. For example, in Fig. 3 (b = 1.5 larger than b = 1.1, shown in Fig. 2), the fully mixed strategy 
is replaced by a stable polarized strategy for which some of the nodes converge on the pure cooperator state and 
some on the pure defector state. Our interpretation of this phenomenon is the following. Fully mixed strategies are 
equilibria that arise in the presence of a balance between the benefit of being connected to a cooperator b and the 
cost of cooperating c. The benefit to cost ratio b/c affects stability of these equilibria, i.e., for a low ratio (b/c) they 
are stable and for a large ratio, they are unstable. 
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IV. EFFECTS OF THE NETWORK TOPOLOGY ON STABILITY 

Fully mixed strategies are interesting equilibria, for which all the players' payoffs are equal. Moreover, different 
from polarized strategies, fully mixed strategy states can be controlled by changing the network connections and/or 
of the parameters of the game. Thus, it is possible that according to the specific application of interest, these may 
be desirable/undesirable configurations. In this section, we analyze how the underlying network structure can affect 
stability of fully mixed strategies. In particular, we analyze how the range of stability _R, defined as the width of the 
6-range (see Fig. 4(a)) associated with stability of the fully mixed state, varies for different network topologies. We 
consider the case of symmetric network topologies, i.e., Aij — Aji, i,j — l,...,N, for which the eigenvalues of 
the matrix B are real (see Appendix B). 



We first analyze Erdos-Renyi random graphs 26j. These are networks for which any two nodes are connected by 
an edge with a constant probability p. Figure 5(a) shows the range of stability R for Erdos-Renyi random graphs of 
200 nodes versus the probability of an edge parameter p. We see that the range of stability increases with p. Notice 
that for networks with low connectivity p, it is unlikely to find a game for which a fully mixed strategy is stable. In 
figure 5(b) we consider Erdos-Renyi graphs of varying dimension N and average degree fixed and equal 20. The figure 
shows the range of stability R versus N. 

We define the degree of node i is defined as fcj = J2j Aij ■ Heterogeneity in the degree distribution is probably the 
most important feature that characterizes the structure of real networks. The discovery that the basic structure of 
many real- world networks is characterized by a power-law degree distribution, was pointed out by Barabasi and Albert 

n 

in their seminal paper 8], and has been verified by many observations of real networks. Specifically, the analysis of 
data sets of biological, social and technological networks has shown that these typically exhibit a power-law degree 
distribution, P{k) ~ k~ v . Networks characterized by a power-law degree distribution are termed scale free. 

Plot 5(c) shows the range of stability R for scale free networks of 200 nodes versus the exponent of the degree 
distribution, 77, for which the average degree is kept fixed at 20. The networks are generated by using the algorithm 
111 Ref. Q. The range of stability increases with r\. This indicates that for networks with high heterogeneity (low 77), 
we are unlikely to find a game (or we are unlikely to be playing a game) for which a fully mixed strategy is stable. 

The form of the degree distribution is an important property of the structure of a network. However, many other 
distinctive properties have been uncovered to characterize the structure of real networks in more detail, such as 
the formation of communities of strongly interconnected nodes, frequently detected in many real networks 
particular forms of correlation or mixing among the network vertices 14 1 . 
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One measure of mixing is the correlation among pairs of linked nodes according to some properties at the network 
nodes. A very simple case is degree correlation [12j . in which vertices choose their neighbors according to their 
respective degrees. Nontrivial forms of degree correlation have been experimentally detected in many real-world 
networks, with social networks being typically characterized by assortative mixing (which is the case where vertices 
are more likely to connect to other vertices with approximately the same degree) and technological and biological 
networks by disassortative mixing (which takes place when connections are more frequent between vertices of different 



degrees). In Ref. 12| degree correlation is measured by means of a single normalized index, the Pearson statistic r 
defined as follows: 

r = — ^ kk'(e kk , - q k qk>), (15) 
a i k,k> 

where q k is the probability that a randomly chosen edge is connected to a node having degree k; a q is the standard 
deviation of the distribution q k and e kk > represents the probability that two vertices at the endpoints of a generic edge 
have degrees k and k' , respectively. Positive values of r indicate assortative mixing, while negative values characterize 
disassortative networks. 

In Figure 5(d) we show the results of a numerical simulation in which we have kept the degree distribution fixed 
(power law with exponent r\ = 3.5, average degree equal 20) and we have made the coefficient r vary from —0.3 to 0.3 
in steps of 0.1 (for more details on the procedures that generates networks with different degree correlation properties, 



see Refs. 
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27]). Specifically, we show that disassortative networks (i.e., r negative) are characterized by a larger 



range of stability for the fully mixed strategy R than their assortative counterparts. 



V. CONCLUSION 



In this paper, we have proposed a fully deterministic nonlinear model of an evolutionary game on a network, for 
which players are allowed to pick a strategy in the interval [0, 1], with corresponding to defection, 1 to cooperation, 
and intermediate values representing mixed strategies in which each player may act as a cooperator or a defector over a 
large number of interactions with a certain probability. Our model is payoff-driven and it does not presume knowledge 
of the other players' strategies. Instead, we consider that strategies evolve based on a comparison of each player's 
payoff with those of his/her neighbors. We remove the unrealistic assumption that all the players are equal and assume 
that their choice of strategy depends on a parameter that takes into account the previous history of the game (in 
terms of payoffs). Under these assumptions, we find that fixed points of the dynamics may correspond to either one 
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FIG. 5: Plot (a) shows the range of stability 7? (shown in Fig. 4(a)) for random Erdos-Renyi graphs of 200 nodes versus the 
probability of an edge parameter p, for 5 different cases (grey points) for each p, with the average R connected by line segments. 
Plot (b) shows the range of stability R for random Erdos-Renyi graphs of dimension N and average degree fixed and equal 20, 
for 10 different cases (grey points) for each N, with the average R connected by line segments. Plot (c) shows the range of 
stability R for scale free networks of 200 nodes versus the exponent of the degree distribution, rj, where the average degree is 
kept fixed at 20. Plot (d) shows the range of stability R for scale free networks of 200 nodes, rj = 3.5, average degree equal 20, 
versus the degree correlation coefficient r. 

of two qualitatively different states: fully mixed strategies, i.e., fixed points for which all the strategies are mixed and 
the payoffs are equal at different nodes, or polarized strategies, i.e., fixed points for which at least one node is a full 
cooperator or defector and the payoffs are unequal at different nodes. We derive a simple condition that relates the 
network structure and the parameters of the game to stability of such fixed points, which provides a unified framework 
to study the effects of different network features, such as random, scale free, and degree-correlated topologies. This 
information can be used to predict which network/game combinations promote mixed versus polarized behavior. 

In our simulations we choose a specific form for the function /, which is consistent with the assumption that 
cooperative behavior arises when the accumulated payoff exceeds a given threshold. Such a choice is supported 
e.g., by empirical studies that have found a positive correlation between the level of cooperation and the payoff for 
players participating in a Prisoner's Dilemma experiment |22|. Also, a natural interpretation of our proposed update 



mechanism is provided by Maslow's motivational theory 



19[ | that assumes that an individual's goals change according 
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to whether certain basic needs are satisfied or not. In the language of game theory this corresponds to assuming 
that individuals start caring about the common good (cooperation) after they have consolidated their individual gain 
(defection). However, our stability analysis is independent of the specific choice of the function / and is valid over a 
broad range of possible functions /. 

We present a stability analysis for fixed points of Eqs. ©. We reduce the high-dimensional stability problem © 
in the low-dimensional form of Eq. ([lip , which depends on the eigenvalues of a relevant matrix. These eigenvalues 
reflect the structure of the underlying network (i.e., the matrix A) and the choice of the parameters of the game 
(i.e., the two scalars b, and c). Similar reductions in a low-dimensional form have been proposed to evaluate (i) the 



stability of the synchronous evolution for networks of coupled oscillators 
state in networks of coupled integrators 



28 



32}, (ii) the stability of the consensus 



33l | , (iii) the stability of discrete state models of genetic control [34| , and (iv) 



35] 



the response of networks of coupled excitable systems to stochastic stimuli 

Our computations provide evidence of the fact that for networks with a heterogeneous degree distribution (scale free) 
or networks with low connectivity, most games played will support the emergence of polarized strategies. However, for 
networks with a homogeneous degree distribution or networks with high connectivity, it is more likely that strategies 
will be mixed. We have also considered the effects of the network degree correlation and found that disassortative 
networks are characterized by a larger range of stability R than assortative networks. Thus if we are given certain 
characteristics of a network (e.g., connectivity, the degree distribution, or the degree correlation), we may be able to 
determine whether polarized or mixed strategies will arise from the dynamics. Alternatively, if we are given a game 
on a network, we may be able to modify the network to preferentially select polarized or mixed strategies depending 
on our application. 

One surprising observation is that making the benefit b larger than the cost c may destabilize the mixed strategy 
state, with some players converging on the pure cooperator and some on the pure defector state. This suggests that 
policies governing dynamics on a network should consider the implications of making the benefit of cooperation too 
high. 

The authors would like to thank Anurag Setty, Zeynep Tukekci, Gregory Taylor, Edward Ott, and Michelle Gir- 
van for insightful discussions. Both authors are indebted to an anonymous reviewer for his valued comments and 
suggestions. 
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Appendix A: The matrix B has at least one zero eigenvalue 

We define (D — A) as the Laplacian matrix L. We note that the sum of the elements of all the rows of the Laplacian 
matrix L = (D — A) is equal zero. It follows that L has at least one eigenvalue equal zero, with associated right 
eigenvector v r = [1, 1, 1] T and left eigenvector v . Moreover, it can be shown that this zero eigenvalue is also the 
only one if the matrix A is irreducible, i.e., the associated digraph is strongly connected. From B T = (bA — cD) T L T , 

rp rp 

we see that B T v e = (bA — cD) T (L T v l ) = and therefore the matrix B has at least one eigenvalue equal zero with 
associated left eigenvector vr. 



Appendix B: A symmetric implies that B has real eigenvalues 

The property of the matrix A of being symmetric does not imply that the matrix B = (D — A)(bA — cD) is 
symmetric. However, it can be shown that if A is symmetric, the spectrum of B is real. In order to do this, we write 
the eigenvalue equation for the matrix B, 

w T (D — A)(bA — cD) — \w T , (Bl) 
where A is a generic eigenvalue (w T the associated left eigenvector). Our goal is to show that A is real. Recall that 



the graph Laplacian L = (D — A) is symmetric and positive scmidcfinitc 
where the matrix Q is also positive semidefinite . This yields, 



. Hence, it can be decomposed as QQ T , 



w T QQ T (bA ~ cD) = Xw T . (B2) 

Right multiplying Eq. (|B2|) by Q, we obtain 

u T Q T (bA - cD)Q = \u T , (B3) 
where u T = w T Q. Since the matrix Q T (bA — cD)Q is symmetric it follows that A is real. 



Appendix C: Stability conditions for b — c 

In the limit in which b = c, the matrix B is equal to B = —b(D — A) 2 = —bL 2 . If the matrix A is symmetric and 
irreducible, it follows that the eigenvalues of the matrix B are negative and that the eigenvector associated with the 
only zero eigenvalue of the matrix B is [1, 1, 1] T . Thus the fixed point associated with this eigenvector is stable, as 
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both conditions (II) and (HIA) presented in the main manuscript, are satisfied. 
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